This is a collection of quantized versions of the inclusionAI/Ling-1T model, developed based on the ik_llama.cpp branch. It offers multiple quantization schemes to meet different memory and performance requirements. These quantized models provide excellent perplexity performance under a given memory footprint and are specifically optimized for the inference efficiency of the MoE architecture.

Natural Language ProcessingGguf

The Ming-flash-omni preview version is a multimodal large model built on the Ling-Flash-2.0 sparse mixture of experts (MoE) architecture, with a total of 100B parameters and only 6B parameters activated per token. This model is a comprehensive upgrade based on Ming-Omni, showing significant improvements in multimodal understanding and generation, especially in speech recognition, image generation, and segmentation editing.

MultimodalDiffusersEnglish

This is the Q4_K_M quantized version of the DeepSeek-MoE-16B model from DeepSeek Company. It enables the MoE (Mixture of Experts) CPU offload function through the Rust binding of llama.cpp, significantly reducing video memory usage.

Natural Language ProcessingGgufMultiple Languages

NVIDIA GPT-OSS-120B Eagle3 is an optimized version based on the OpenAI gpt-oss-120b model. It adopts the Mixture of Experts (MoE) architecture, with a total of 120 billion parameters and 5 billion active parameters. This model supports both commercial and non-commercial use and is suitable for text generation tasks, especially for the development of AI Agent systems, chatbots, and other applications.

Natural Language ProcessingSafetensors

This is the GGUF format version of the GPT-OSS 20B model based on WeOpenML, which first implements the innovative MoE CPU expert offloading technology. This technology achieves a 99.9% reduction in video memory while maintaining the full generation quality, and only requires 2MB of video memory to run a 20 billion parameter mixture of experts model.

Natural Language ProcessingSafetensorsMultiple Languages

GLM-4.6-AWQ is a model that performs high-performance AWQ quantization on Zhipu AI's GLM-4.6 (357B MoE). It is optimized for vLLM inference and can effectively improve the throughput of production deployment. This model uses 4-bit quantization technology, which significantly reduces video memory usage while maintaining high accuracy.

Natural Language ProcessingTransformersMultiple Languages

Wan2.2 is a significant upgrade version of the basic video model. It introduces the Mixture of Experts (MoE) architecture, incorporates carefully curated aesthetic data, and is trained on larger datasets to enhance the ability to generate complex motions. This model supports generating 5-second videos with 480P and 720P resolutions, and has significant improvements in video generation quality and performance.

MultimodalDiffusers

The 5-bit quantized version of the IBM Granite-4.0-H-Tiny model, optimized for Apple silicon chips. It uses a hybrid architecture of Mamba-2 and soft attention, combined with a Mixture of Experts model (MoE), to achieve efficient inference while maintaining high quality.

Natural Language ProcessingMlxMultiple Languages

IBM Granite-4.0-H-Tiny is a hybrid Mamba-2/Transformer model optimized for Apple silicon chips, using 3-bit quantization technology, and is designed for long context, efficient inference, and enterprise use. This model combines the Mamba-2 architecture with the Mixture of Experts (MoE) technology, significantly reducing memory usage while maintaining expressiveness.

Natural Language ProcessingMlxMultiple Languages

Qwen3-Next-80B-A3B-Instruct is an efficient sparse mixture of experts model developed by the Tongyi Qianwen team. It has a total of 80 billion parameters, with only 3 billion active parameters. The model adopts an innovative hybrid attention mechanism and a MoE architecture with an extremely low activation rate, significantly improving inference efficiency while maintaining strong performance. It natively supports a context length of 262,000 tokens and can be extended to 1 million tokens.

Natural Language ProcessingTransformers

Megrez 2-3x7B-A3B is a device-native large language model that adopts the Mixture of Experts (MoE) architecture, combining the accuracy of MoE and the compactness of dense models. The model is trained on 8T token data, has a context length of 32K, and plans to improve inference and agent capabilities in the future.

Natural Language ProcessingGgufMultiple Languages

Qwen3-Next-80B-A3B-Thinking-AWQ-4bit is a 4-bit version generated from the Qwen3-Next-80B-A3B-Thinking model through the AWQ quantization technology, which significantly improves the model's inference efficiency. This model adopts a hybrid attention mechanism and a highly sparse MoE architecture, and performs excellently in complex reasoning tasks. It supports a context length of up to 262K tokens.

Natural Language ProcessingTransformers

Qwen3-Next-80B-A3B-Instruct-AWQ-4bit is an efficient version based on the Qwen3-Next-80B-A3B-Instruct model with 4-bit AWQ quantization. This model adopts a hybrid attention mechanism and a highly sparse mixture-of-experts (MoE) architecture, supporting a context length of up to 262K tokens. It significantly reduces the computational resource requirements while maintaining high performance.

Natural Language ProcessingTransformers

LLaDA-MoE is a new type of mixture-of-experts language model constructed based on the diffusion principle. It is the first open-source MoE diffusion large language model, pre-trained from scratch on approximately 20 trillion tokens. It has a total of 7 billion parameters, and only 1.4 billion parameters are activated during inference. It performs excellently in tasks such as code generation and mathematical reasoning.

Natural Language ProcessingTransformers

Ring-mini-2.0 is a high-performance inference MoE model deeply optimized based on the Ling 2.0 architecture. It has only 16 billion total parameters and 1.4 billion active parameters, yet it achieves comprehensive inference capabilities comparable to dense models below 10 billion scale. It performs excellently in logical reasoning, code generation, and math tasks, supports processing long contexts of up to 128,000 tokens, and enables high-speed generation of over 300 tokens per second.

Natural Language ProcessingSafetensors

Ling 2.0 is a large language model family based on the MoE architecture. The first version, Ling-mini-2.0, has a compact parameter scale but powerful functions. It performs excellently in complex reasoning and instruction following. It adopts the MoE architecture with a 1/32 activation rate, achieving a perfect combination of excellent performance and efficient operation.

Natural Language ProcessingTransformers

Klear is a large language model of Sparse Mixture of Experts (MoE) developed by the Kwai-Klear team of Kuaishou. It features high performance and inference efficiency. With a total of 46 billion parameters and only 2.5 billion activated parameters, it performs excellently in multiple capability benchmark tests, providing an efficient and powerful solution for practical applications.

Natural Language ProcessingTransformersMultiple Languages

Wan2.2-S2V-14B is a Mixture of Experts (MoE) model designed for audio-driven cinema-level video generation. It can generate high-quality video content based on input audio, reference images, and text prompts, supports 480P and 720P resolutions, and features complex motion generation and cinema-level aesthetic effects.

MultimodalDiffusersSpanish

TSLAM-15B is a 15-billion parameter telecom-specific language model developed by NetoAI Solutions. It is based on the Mixture-of-Experts (MoE) architecture and optimized for telecom domain expertise, advanced reasoning, and action-oriented workflows.

Natural Language ProcessingSafetensors

This is a quantized version of a 6-billion parameter Mixture of Experts (MoE) model based on the GPT architecture. After pruning optimization, only 7 experts are retained. It focuses on text generation tasks and is committed to making knowledge accessible to everyone.

Natural Language ProcessingGguf